Electronic device and method using machine learning for identifying characteristics of users located within a specific space

ABSTRACT

Electronic device using intelligent analysis for identifying characteristics of users located within a specific space. The electronic device includes a controller configured to identify each of a plurality of users located within a specific space, and generate a control command for controlling operation of at least one device associated with the specific space based on characteristic information related to each of the identified plurality of users.

CROSS-REFERENCE TO RELATED APPLICATIONS

Pursuant to 35 U.S.C. § 119(a), this application claims the benefit ofearlier filing date and right of priority to Korean Application No.10-2017-0029576, filed on Mar. 8, 2017, the contents of which areincorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to smart device technology, and moreparticularly, to an electronic device using machine learning foridentifying characteristics of users located within a specific space.

2. Background of the Invention

Advancements have been made in context awareness technology whichincludes devices able to recognize or predict a user's needs based onhistorical information, a user's environment, context of a presentstatus of a user, or a user's particular schedule, and the like, usingmachine learning, data mining, pattern recognition, and otherintelligent algorithms and technologies.

With the development of the foregoing context awareness technologies,there is an increasing demand for an improved terminal capable ofaccurately and effectively performing a function suitable for a user'scurrent or upcoming environment or context.

Recently, with development of Internet of Things (IoT), technologies arebeing developed that provide environments optimized for users throughcommunication links between objects (things). As a part of thistechnology development, the present disclosure proposes a method ofproviding an environment optimized for a user by performing a moreorganic control between objects in a manner of utilizing artificialintelligence and other techniques.

SUMMARY OF THE DISCLOSURE

Therefore, an aspect of the detailed description is to provide a spaceenvironment optimized for a user.

Another aspect of the detailed description is to provide an environmentthat meets a use purpose of a space (i.e., space use purpose) to aplurality of members using the space.

Disclosed here is an electronic device capable of controlling at leastone device installed in a specific space, the electronic deviceincluding an learning data unit configured to recognize a user locatedwithin the specific space, and generate a control command forcontrolling an operation of the at least one device installed in thespecific space, based on characteristic information related to therecognized user, and a controller configured to control the operation ofthe at least one device based on the control command generated in theartificial intelligence unit, wherein the learning data unit sets adriving condition of the at least one device by combining characteristicinformation related to each of a plurality of users when the pluralityof user are located within the specific space.

In one embodiment disclosed herein, the learning data unit may predict ause purpose of the specific space based on the combination of thecharacteristic information related to each of the plurality of userslocated within the specific space, and set the driving condition of theat least one device to meet the predicted use purpose.

In one embodiment disclosed herein, a different driving command may beset for the at least one device according to a different use purpose ofthe specific space, and the learning data unit may set the drivingcondition of the at least one device using the driving command setaccording to the predicted use purpose.

In one embodiment disclosed herein, the learning data unit may extractcommon elements from the characteristic information regarding each ofthe plurality of users, and predict the use purpose of the specificspace based on the extracted common elements.

In one embodiment disclosed herein, the learning data unit may learn thecharacteristic information regarding each of the plurality of usersbased on a machine learning technology.

In one embodiment disclosed herein, the characteristic informationregarding the user may include at least one of biometric information,behavior information, log information related to the specific space, andcompanion information located together within the specific space.

In one embodiment disclosed herein, the electronic device may furtherinclude a communication unit configured to execute communication with anexternal device, and the learning data unit may predict a plurality ofuser to be located within the specific space on the basis of a messagereceived from the external device through the communication unit.

In one embodiment disclosed herein, the learning data unit may receive aplurality of schedule information stored in a plurality of electronicdevices through the communication unit, and the learning data unit maygenerate schedule information related to the specific space based on thereceived plurality of schedule information.

In one embodiment disclosed herein, the learning data unit may set thedriving condition of the at least one device installed in the specificspace, based on a combination of the characteristic information relatedto the plurality of users, so as to meet the schedule informationrelated to the specific space, when the schedule information related tothe specific space is generated.

In one embodiment disclosed herein, the electronic device may furtherinclude a camera configured to capture an image of the specific space,and the learning data unit may analyze the captured image of thespecific space based on an image analysis algorithm, and detect aplurality of users located within the specific space using the analysisresult.

An electronic device assisting driving of a vehicle according to anotherembodiment may include an learning data unit configured to learnboarding status information related to passengers seated in the vehicle,and a controller configured to control the vehicle based on the learnedboarding status information, wherein the learning data unit monitorssituation information related to the vehicle, and predicts a generationof a boarding event based on at least one of the learned boarding statusinformation and the monitored situation information, and wherein thelearning data unit extracts at least one control command to drive thevehicle from the learned boarding status information when the generationof the boarding event is predicted.

In one embodiment disclosed herein, the controller may control thevehicle based on the at least one control command extracted by theartificial intelligence unit.

In one embodiment disclosed herein, the boarding event may be an eventthat a passenger expected to be seated in the vehicle boards the vehicleat an expected boarding time.

In one embodiment disclosed herein, the learning data unit may set seatinformation to be seated by each of a plurality of passengers, based onthe boarding status information related to each of the plurality ofpassengers, when a boarding event that the plurality of passengers boardthe vehicle is predicted.

In one embodiment disclosed herein, the controller may execute a seatposture control set for each of the plurality of passengers, based onthe boarding status information related to each of the plurality ofpassengers.

In other embodiments, an electronic device includes a controllerconfigured to: identify each of a plurality of users located within aspecific space; and generate a control command for controlling operationof at least one device associated with the specific space based oncharacteristic information related to each of the identified pluralityof users.

A further embodiment includes an electronic device for assisting indriving of a vehicle. The electronic device includes a memory and acontroller configured to store, in the memory, boarding statusinformation related to passengers; monitor situation information relatedto the vehicle; control the vehicle based on the learned boarding statusinformation; predict a boarding event based on at least one of theboarding status information and the monitored situation information; andgenerating a control command for driving the vehicle based on theboarding status information and the predicting of the boarding event.

Further scope of applicability of the present application will becomemore apparent from the detailed description given hereinafter. However,it should be understood that the detailed description and specificexamples, while indicating preferred embodiments of the invention, aregiven by way of illustration only, since various changes andmodifications within the spirit and scope of the invention will becomeapparent to those skilled in the art from the detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andtogether with the description serve to explain the principles of theinvention.

FIG. 1 is a block diagram of an electronic device in accordance with oneexemplary embodiment of the present invention.

FIG. 2 is a flowchart illustrating a method of controlling a specificspace for providing an environment optimized for a user.

FIG. 3 illustrates the control method of FIG. 2.

FIG. 4 is a flowchart illustrating a method of controlling a specificspace when a plurality of users are located in the specific space.

FIGS. 5A and 5B illustrate the control method of FIG. 4.

FIGS. 6A(a), 6A(b), 6A(c), 6A(d), 6B(a), 6B(b), 6C illustrate a methodof recognizing a plurality of users who use a specific space.

FIGS. 7A, 7B illustrate a method of generating schedule informationrelated to a specific space by combining schedule information regardinga plurality of users.

FIG. 8 is a flowchart illustrating a method of controlling a vehicleusing an electronic device.

FIGS. 9 to 11 and 12A, 12B and 12C illustrate the control method of FIG.8.

FIGS. 13A(a), 13A(b), 13B, and 13C illustrate methods of providingpassenger seat information to passengers in accordance with oneembodiment of the present invention.

FIGS. 14A(a), 14A(b), 14A(c), 14B(a), 14B(b), 14B(c) illustrate methodsof predicting passengers included in a boarding event.

FIGS. 15A, 15B, 15C illustrate a method of reproducing content whiledriving a vehicle with a plurality of passengers seated.

FIGS. 16A(a), 16A(b), 16B(a), 16B(b) illustrate methods of controllingan environment of a vehicle during driving of the vehicle.

FIGS. 17A(a), 17A(b), 17A(c), 17B(a), 17B(b), 17B(c) illustrateembodiments of controlling a vehicle when an error occurs in predictionof a boarding event.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Description will now be given in detail according to exemplaryembodiments disclosed herein, with reference to the accompanyingdrawings. For the sake of brief description with reference to thedrawings, the same or equivalent components may be provided with thesame reference numbers, and description thereof will not be repeated. Ingeneral, a suffix such as “module” and “unit” may be used to refer toelements or components. Use of such a suffix herein is merely intendedto facilitate description of the specification, and the suffix itself isnot intended to give any special meaning or function. In the presentdisclosure, that which is well-known to one of ordinary skill in therelevant art has generally been omitted for the sake of brevity. Theaccompanying drawings are used to help easily understand varioustechnical features and it should be understood that the embodimentspresented herein are not limited by the accompanying drawings. As such,the present disclosure should be construed to extend to any alterations,equivalents and substitutes in addition to those which are particularlyset out in the accompanying drawings.

It will be understood that although the terms first, second, etc. may beused herein to describe various elements, these elements should not belimited by these terms. These terms are generally only used todistinguish one element from another.

It will be understood that when an element is referred to as being“connected with” another element, the element can be directly connectedwith the other element or intervening elements may also be present. Incontrast, when an element is referred to as being “directly connectedwith” another element, there are no intervening elements present.

A singular representation may include a plural representation unless itrepresents a definitely different meaning from the context. Terms suchas “include” or “has” are used herein and should be understood that theyare intended to indicate an existence of several components, functionsor steps, disclosed in the specification, and it is also understood thatgreater or fewer components, functions, or steps may likewise beutilized.

Terminals presented herein may be implemented using a variety ofdifferent types of terminals. Examples of such terminals includecellular phones, smart phones, user equipment, laptop computers, digitalbroadcast terminals, personal digital assistants (PDAs), portablemultimedia players (PMPs), navigators, portable computers (PCs), slatePCs, tablet PCs, ultra-books, wearable devices (for example, smartwatches, smart glasses, head mounted displays (HMDs)), and the like.

By way of non-limiting example only, further description will be madewith reference to particular types of terminals. However, such teachingsapply equally to other types of terminals, such as those types notedherein. In addition, these teachings may also be applied to stationaryterminals such as digital TV, desktop computers, and the like.

The terminal 100 may include components, such as a wirelesscommunication unit 110, an input unit 120, learning data unit 130, asensing unit 140, an output unit 150, an interface unit 160, a memory170, a controller 180, a power supply unit 190 and the like. FIG. 1illustrates the terminal having various components, but it is understoodthat implementing all of the illustrated components is not arequirement, and that greater or fewer components may alternatively beimplemented.

In more detail, the wireless communication unit 110 of those componentsmay typically include one or more modules which permit wirelesscommunications between the terminal 100 and a wireless communicationsystem, between the terminal 100 and another terminal 100, or betweenthe terminal 100 and an external server.

The wireless communication unit 110 may include at least one of abroadcast receiving module 111, a mobile communication module 112, awireless Internet module 113, a short-range communication module 114, alocation information module 115 and the like.

The input unit 120 may include a camera 121 for inputting an imagesignal, a microphone 122 or an audio input module for inputting an audiosignal, or a user input unit 123 (for example, a touch key, a push key(or a mechanical key), etc.) for allowing a user to input information.Audio data or image data collected by the input unit 120 may be analyzedand processed by a user's control command.

The learning data unit 130 may be configured to receive, categorize,store, and output information to be utilized for data mining, dataanalysis, intelligent decision making, and machine learning algorithmsand techniques. The learning data unit 130 may include one or morememory units configured to store data that is received, detected,sensed, generated, predefined, or otherwise output by the terminal, orreceived, detected, sensed, generated, predefined, or otherwise outputby another component, device, terminal, or entity in communication withthe terminal.

The learning data unit 130 may include memory incorporated orimplemented at the terminal. In some embodiments, learning data unit 130may be implemented using memory 170. Alternatively or additionally, thelearning data unit 130 may be implemented using memory associated withthe terminal, such as an external memory directly coupled to theterminal or memory maintained at a server in communication with theterminal. In other embodiments, the learning data unit 130 may beimplemented using memory maintained in a cloud computing environment, orother remote memory location that is accessible by the terminal througha communication scheme, such as a network.

The learning data unit 130 is generally configured to store data in oneor more databases to identify, index, categorize, manipulate, store,retrieve, and output the data for use in supervised or unsupervisedlearning, data mining, predictive analytics, or other machine learningtechniques. The information stored at the learning data unit 130 may beutilized by the controller 180, or one or more other controllers of theterminal, using any of a variety of different types of data analysis andmachine learning algorithms and techniques. Examples of such algorithmsand techniques include k-Nearest neighbor systems, fuzzy logic (e.g.,possibility theory), neural networks, boltzmann machines, vectorquantization, pulsed neural nets, support vector machines, maximummargin classifiers, hill-climbing, inductive logic systems, bayesiannetworks, petri nets (e.g., finite state machines, mealy machines, moorefinite state machines), classifier trees (e.g., perceptron trees,support vector trees, markov trees, decision tree forests, randomforests), pandemonium models and systems, clustering, artificiallyintelligent planning, artificially intelligent forecasting, data fusion,sensor fusion, image fusion, reinforcement learning, augmented reality,pattern recognition, automated planning, and the like.

The controller 180 may request, retrieve, receive, or otherwise utilizethe data of the learning data unit 130 to determine or predict at leastone executable operation of the terminal based on the informationdetermined or generated using the data analysis and machine learningalgorithms and techniques, and control the terminal to execute apredicted or desired operation among the at least one executableoperation. The controller 180 may perform various functions implementingemulation of intelligence (i.e., knowledge based systems, reasoningsystems, and knowledge acquisition systems); and including systems forreasoning with uncertainty (e.g., fuzzy logic systems), adaptivesystems, machine learning systems, artificial neural networks, and thelike.

The controller 180 may also include sub-modules to enable itsperformance and/or execution involving voice and natural speech languageprocessing, such as an I/O processing module, environment conditionmodule, a speech-to-text (STT) processing module, a natural languageprocessing module, a task flow processing module, and a serviceprocessing module. Each of these sub-modules may also have access to oneor more systems or data and models at the terminal, or a subset orsuperset thereof, including scheduling, vocabulary index, user data,task flow models, service models, and automatic speech recognition (ASR)systems. In other embodiments, the controller 180 or other aspects ofthe terminal may be implemented with said sub-modules, systems, or dataand models.

In some examples, based on the data at the learning data unit 130, thecontroller 180 may be configured to perform detecting and sensing a needbased on a contextual condition or a user's intent expressed in a userinput or natural language input; actively eliciting and/or obtaininginformation needed to fully determine a need based on the contextualcondition or a user's intent (e.g., by analyzing historical dataincluding historical input and output, pattern matching, disambiguatingwords, input intentions, etc.); determining the task flow for executinga function in response to the need based on the contextual condition oruser's intent; and executing the task flow to meet the need based on thecontextual condition or user's intent.

In some embodiments, the controller 180 may implement specific hardwareelements dedicated for learning data processes including memistors,memristors, transconductance amplifiers, pulsed neural circuits,artificially intelligent nanotechnology systems (e.g., autonomousnanomachines) or artificially intelligent quantum mechanical systems(e.g., quantum neural networks), and the like. In some embodiments, thecontroller 180 may include pattern recognition systems such as machinevision systems, acoustic recognition systems, handwriting recognitionsystems, data fusion systems, sensor fusion systems, and soft sensors.Machine vision systems can also include content based image retrieval,optical character recognition, augmented reality, egomotion, tracking oroptical flow, and the like.

The controller 180 may be configured to collect, sense, monitor,extract, detect, and/or receive signals or data, via one or more sensingcomponents at the terminal, in order to collect information forprocessing and storage at the learning data unit 130 and for use in dataanalysis and machine learning operations. Collection of information mayinclude sensing information through a sensor, extracting informationstored in the memory, such as memory 170, or receiving information fromanother terminal, entity, or an external storage through communicationmeans. Thus in one example, the controller 180 may collect historicalusage information at the terminal, store the historical usageinformation for use in data analytics, and at a future occurrence,determine a best match for executing a particular function usingpredictive modeling based on the stored historical usage information.

The controller 180 may also receive or sense information of thesurrounding environment, or other information, through the sensing unit140. In addition, the controller 180 may receive broadcast signalsand/or broadcast-related information, wireless signals, wireless data,and the like through the wireless communication unit 110. The controller180 may also receive image information (or a corresponding signal),audio information (or a corresponding signal), data, or user-inputinformation from an input unit.

The controller 180 may collect information in real time, and process orcategorize the information (for example, in a knowledge graph, commandpolicy, personalization database, dialog engine, etc.), and store theprocessed information in the memory 170 or the learning data unit 130.

When the operation of the terminal is determined based on data analysisand machine learning algorithms and techniques, the controller 180 maycontrol the components of the terminal to execute the determinedoperation. The controller 180 may then execute the determined operationby controlling the terminal based on the control command.

In some embodiments, when a specific operation is executed, thecontroller 180 may analyze history information indicating the executionof the specific operation through data analysis and machine learningalgorithms and techniques and execute updating of previously-learnedinformation based on the analyzed information. Accordingly, thecontroller 180, in combination with the learning data unit 130, canimprove the accuracy of future performance of the data analysis andmachine learning algorithms and techniques based on the updatedinformation.

The sensing unit 140 may include at least one sensor which senses atleast one of internal information of the terminal, a surroundingenvironment of the terminal and user information. For example, thesensing unit 140 may include a proximity sensor 141, an illuminationsensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, aG-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, aninfrared (IR) sensor, a finger scan sensor, a ultrasonic sensor, anoptical sensor (for example, refer to the camera 121), a microphone 122,a battery gage, an environment sensor (for example, a barometer, ahygrometer, a thermometer, a radiation detection sensor, a thermalsensor, a gas sensor, etc.), and a chemical sensor (for example, anelectronic nose, a health care sensor, a biometric sensor, etc.). On theother hand, the terminal disclosed herein may utilize information insuch a manner of combining information sensed by at least two sensors ofthose sensors.

The output unit 150 may be configured to output an audio signal, a videosignal or a tactile signal. The output unit 150 may include a displayunit 151, an audio output unit 152, a haptic module 153, an opticaloutput unit 154 and the like. The display unit 151 may have aninter-layered structure or an integrated structure with a touch sensorso as to implement a touch screen. The touch screen may provide anoutput interface between the terminal 100 and a user, as well asfunctioning as the user input unit 123 which provides an input interfacebetween the terminal 100 and the user.

The interface unit 160 may serve as an interface with various types ofexternal devices connected with the terminal 100. The interface unit160, for example, may include wired or wireless headset ports, externalpower supply ports, wired or wireless data ports, memory card ports,ports for connecting a device having an identification module, audioinput/output (I/O) ports, video I/O ports, earphone ports, or the like.The terminal 100 may execute an appropriate control associated with aconnected external device, in response to the external device beingconnected to the interface unit 160.

The memory 170 may store a plurality of application programs (orapplications) executed in the terminal 100, data for operations of theterminal 100, instruction words, and the like. At least some of thoseapplication programs may be downloaded from an external server viawireless communication. Some others of those application programs may beinstalled within the terminal 100 at the time of being shipped for basicfunctions of the terminal 100 (for example, receiving a call, placing acall, receiving a message, sending a message, etc.). On the other hand,the application programs may be stored in the memory 170, installed inthe terminal 100, and executed by the controller 180 to perform anoperation (or a function) of the terminal 100.

The controller 180 may typically control an overall operation of theterminal 100 in addition to the operations associated with theapplication programs. The controller 180 may provide or processinformation or functions appropriate for a user in a manner ofprocessing signals, data, information and the like, which are input oroutput by the aforementioned components, or activating the applicationprograms stored in the memory 170.

Terminal 100 is shown implemented with one controller 180 facilitatingoperation of all of the various units (e.g., wireless communication unit110, input unit 120, learning data unit 130, sensing unit 140, outputunit 150, interface unit 160, etc.) and submodules shown in the figure.However, one or more separate controllers 180 may alternatively beimplemented for any or all of such units and submodules.

Furthermore, the controller 180 may control at least part of thecomponents illustrated in FIG. 1, in order to drive the applicationprograms stored in the memory 170. In addition, the controller 180 maydrive the application programs by combining at least two of thecomponents included in the terminal 100 for operation.

The power supply unit 190 may receive external power or internal powerand supply appropriate power required for operating respective elementsand components included in the terminal 100 under the control of thecontroller 180. The power supply unit 190 may include a battery, and thebattery may be an embedded battery or a replaceable battery.

At least part of those elements and components may be combined toimplement operation and control of the terminal or a control method ofthe terminal according to various exemplary embodiments describedherein. Also, the operation and control or the control method of theterminal may be implemented in the terminal in such a manner ofactivating at least one application program stored in the memory 170.

Hereinafter, each aforementioned component will be described in moredetail with reference to FIG. 1, prior to explaining various exemplaryembodiments implemented by the terminal 100 having the configuration.

First, the wireless communication unit 110 will be described. Thebroadcast receiving module 111 of the wireless communication unit 110may receive a broadcast signal and/or broadcast associated informationfrom an external broadcast managing entity via a broadcast channel. Thebroadcast channel may include a satellite channel and a terrestrialchannel. At least two broadcast receiving modules 111 may be provided inthe terminal 100 to simultaneously receive at least two broadcastchannels or switch the broadcast channels.

The mobile communication module 112 may transmit/receive wirelesssignals to/from at least one of network entities, for example, a basestation, an external terminal, a server, and the like, on a mobilecommunication network, which is constructed according to technicalstandards or transmission methods for mobile communications (forexample, Global System for Mobile Communication (GSM), Code DivisionMulti Access (CDMA), Wideband CDMA (WCDMA), High Speed Downlink Packetaccess (HSDPA), Long Term Evolution (LTE), and the like). The wirelesssignals may include audio call signal, video (telephony) call signal, orvarious formats of data according to transmission/reception oftext/multimedia messages.

The wireless Internet module 113 denotes a module for wireless Internetaccess. This module may be internally or externally coupled to theterminal 100. The wireless Internet module 113 may transmit/receivewireless signals via communication networks according to wirelessInternet technologies. Examples of such wireless Internet access mayinclude Wireless LAN (WLAN), Wireless Fidelity (Wi-Fi) Direct, DigitalLiving Network Alliance (DLNA), Wireless Broadband (Wibro), WorldwideInteroperability for Microwave Access (Wimax), High Speed DownlinkPacket Access (HSDPA), Long Term Evolution (LTE), and the like. Thewireless Internet module 113 may transmit/receive data according to atleast one wireless Internet technology within a range including evenInternet technologies which are not aforementioned.

From the perspective that the wireless Internet accesses according toWibro, HSDPA, GSM, CDMA, WCDMA, LET and the like are executed via amobile communication network, the wireless Internet module 113 whichperforms the wireless Internet access via the mobile communicationnetwork may be understood as a type of the mobile communication module112.

The short-range communication module 114 denotes a module forshort-range communications. Suitable technologies for implementing theshort-range communications may include BLUETOOTH™, Radio FrequencyIDentification (RFID), Infrared Data Association (IrDA), Ultra-WideBand(UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity(Wi-Fi), Wi-Fi Direct, and the like. The short-range communicationmodule 114 may support wireless communications between the terminal 100and a wireless communication system, between the terminal 100 andanother terminal 100, or between the terminal and a network whereanother terminal 100 (or an external server) is located, via wirelesspersonal area networks.

Here, the another terminal 100 may be a wearable device, for example, asmart watch, smart glasses or a head mounted display (HMD), which isable to exchange data with the terminal 100 (or to like data with theterminal 100). The short-range communication module 114 may sense(recognize) a wearable device, which is able to communicate with theterminal), near the terminal 100. In addition, when the sensed wearabledevice is a device which is authenticated to communicate with theterminal 100 according to the present disclosure, the controller 180 maytransmit at least part of data processed in the terminal 100 to thewearable device via the short-range communication module 114. Hence, auser of the wearable device may use the data processed in the terminal100 on the wearable device. For example, when a call is received in theterminal 100, the user may answer the call using the wearable device.Also, when a message is received in the terminal 100, the user may checkthe received message using the wearable device.

The location information module 115 denotes a module for detecting orcalculating a position of the terminal. An example of the locationinformation module 115 may include a Global Position System (GPS) moduleor a Wi-Fi module. For example, when the terminal uses the GPS module, aposition of the terminal may be acquired using a signal sent from a GPSsatellite. As another example, when the terminal uses the Wi-Fi module,a position of the terminal may be acquired based on information relatedto a wireless access point (AP) which transmits or receives a wirelesssignal to or from the Wi-Fi module. According to the need, the locationinformation module 115 may perform any function of the other modules ofthe wireless communication unit 110 to obtain data on the location ofthe terminal. As a module used to acquire the location (or currentlocation) of the terminal, the location information module 115 may notbe necessarily limited to a module for directly calculating or acquiringthe location of the terminal.

Next, the input unit 120 may be configured to provide an audio or videosignal (or information) input to the terminal or information input by auser to the terminal. For the input of the audio information, theterminal 100 may include one or a plurality of cameras 121. The camera121 may process image frames of still pictures or video obtained byimage sensors in a video call mode or a capture mode. The processedimage frames may be displayed on the display unit 151. On the otherhand, the plurality of cameras 121 disposed in the terminal 100 may bearranged in a matrix configuration. By use of the cameras 121 having thematrix configuration, a plurality of image information having variousangles or focal points may be input into the terminal 100. Also, theplurality of cameras 121 may be arranged in a stereoscopic structure toacquire a left image and a right image for implementing a stereoscopicimage.

The microphone 122 may process an external audio signal into electricaudio data. The processed audio data may be utilized in various mannersaccording to a function being executed in the terminal 100 (or anapplication program being executed). On the other hand, the microphone122 may include assorted noise removing algorithms to remove noisegenerated in the course of receiving the external audio signal.

The user input unit 123 may receive information input by a user. Wheninformation is input through the user input unit 123, the controller 180may control an operation of the terminal 100 to correspond to the inputinformation. The user input unit 123 may include a mechanical inputelement (or a mechanical key, for example, a button located on afront/rear surface or a side surface of the terminal 100, a dome switch,a jog wheel, a jog switch, etc.), and a touch-sensitive input means. Asone example, the touch-sensitive input means may be a virtual key, asoft key or a visual key, which is displayed on a touch screen throughsoftware processing, or a touch key which is disposed on a portionexcept for the touch screen. On the other hand, the virtual key or thevisual key may be displayable on the touch screen in various shapes, forexample, graphic, text, icon, video or a combination thereof.

On the other hand, the sensing unit 140 may sense at least one ofinternal information of the terminal, surrounding environmentinformation of the terminal and user information, and generate a sensingsignal corresponding to it. The controller 180 may control an operationof the terminal 100 or execute data processing, a function or anoperation associated with an application program installed in theterminal based on the sensing signal. Hereinafter, description will begiven in more detail of representative sensors of various sensors whichmay be included in the sensing unit 140.

First, a proximity sensor 141 refers to a sensor to sense presence orabsence of an object approaching a surface to be sensed, or an objectdisposed near a surface to be sensed, by using an electromagnetic fieldor infrared rays without a mechanical contact. The proximity sensor 141may be arranged at an inner region of the terminal covered by the touchscreen, or near the touch screen. The proximity sensor 141 may have alonger lifespan and a more enhanced utility than a contact sensor.

The proximity sensor 141, for example, may include a transmissive typephotoelectric sensor, a direct reflective type photoelectric sensor, amirror reflective type photoelectric sensor, a high-frequencyoscillation proximity sensor, a capacitance type proximity sensor, amagnetic type proximity sensor, an infrared rays proximity sensor, andso on. When the touch screen is implemented as a capacitance type, theproximity sensor 141 may sense proximity of a pointer to the touchscreen by changes of an electromagnetic field, which is responsive to anapproach of an object with conductivity. In this case, the touch screen(touch sensor) may be categorized as a proximity sensor.

On the other hand, for the sake of brief explanation, a state that thepointer is positioned to be proximate onto the touch screen withoutcontact will be referred to as ‘proximity touch,’ whereas a state thatthe pointer substantially comes in contact with the touch screen will bereferred to as ‘contact touch.’ For the position corresponding to theproximity touch of the pointer on the touch screen, such position willcorrespond to a position where the pointer faces perpendicular to thetouch screen upon the proximity touch of the pointer. The proximitysensor 141 may sense proximity touch, and proximity touch patterns(e.g., distance, direction, speed, time, position, moving state, etc.).On the other hand, the controller 180 may process data (or information)corresponding to the proximity touches and the proximity touch patternssensed by the proximity sensor 141, and output visual informationcorresponding to the process data on the touch screen. In addition, thecontroller 180 may control the terminal 100 to execute differentoperations or process different data (or information) according towhether a touch with respect to the same point on the touch screen iseither a proximity touch or a contact touch.

A touch sensor may sense a touch (or touch input) applied onto the touchscreen (or the display unit 151) using at least one of various types oftouch methods, such as a resistive type, a capacitive type, an infraredtype, a magnetic field type, and the like.

As one example, the touch sensor may be configured to convert changes ofpressure applied to a specific part of the display unit 151 or acapacitance occurring from a specific part of the display unit 151, intoelectric input signals. Also, the touch sensor may be configured tosense not only a touched position and a touched area, but also touchpressure. Here, a touch object is an object to apply a touch input ontothe touch sensor. Examples of the touch object may include a finger, atouch pen, a stylus pen, a pointer or the like.

When touch inputs are sensed by the touch sensors as described above,corresponding signals may be transmitted to a touch controller. Thetouch controller may process the received signals, and then transmitcorresponding data to the controller 180. Accordingly, the controller180 may sense which region of the display unit 151 has been touched.Here, the touch controller may be a component separate from thecontroller 180 or the controller 180 itself.

On the other hand, the controller 180 may execute a different control orthe same control according to a type of an object which touches thetouch screen (or a touch key provided in addition to the touch screen).Whether to execute the different control or the same control accordingto the object which gives a touch input may be decided based on acurrent operating state of the terminal 100 or a currently executedapplication program.

Meanwhile, the touch sensor and the proximity sensor may be executedindividually or in combination, to sense various types of touches, suchas a short (or tap) touch, a long touch, a multi-touch, a drag touch, aflick touch, a pinch-in touch, a pinch-out touch, a swipe touch, ahovering touch, and the like.

An ultrasonic sensor may be configured to recognize position informationrelating to a sensing object by using ultrasonic waves. The controller180 may calculate a position of a wave generation source based oninformation sensed by an illumination sensor and a plurality ofultrasonic sensors. Since light is much faster than ultrasonic waves, atime for which the light reaches the optical sensor may be much shorterthan a time for which the ultrasonic wave reaches the ultrasonic sensor.The position of the wave generation source may be calculated using thisfact. In more detail, the position of the wave generation source may becalculated by using a time difference from the time that the ultrasonicwave reaches the sensor based on the light as a reference signal.

The camera 121 of the input unit 120 may be a type of camera sensor. Thecamera sensor may include at least one of a photo sensor and a lasersensor. The camera 121 and the laser sensor may be combined to detect atouch of the sensing object with respect to a 3D stereoscopic image. Thephoto sensor may be laminated on the display device. The photo sensormay be configured to scan a movement of the sensing object in proximityto the touch screen. In more detail, the photo sensor may include photodiodes and transistors at rows and columns to scan content placed on thephoto sensor by using an electrical signal which changes according tothe quantity of applied light. Namely, the photo sensor may calculatethe coordinates of the sensing object according to variation of light tothus obtain position information of the sensing object.

The display unit 151 may output information processed in the terminal100. For example, the display unit 151 may display execution screeninformation of an application program driven in the terminal 100 or userinterface (UI) and graphic user interface (GUI) information in responseto the execution screen information.

Furthermore, the display unit 151 may also be implemented as astereoscopic display unit for displaying stereoscopic images. Thestereoscopic display unit may employ a stereoscopic display scheme suchas stereoscopic scheme (a glass scheme), an auto-stereoscopic scheme(glassless scheme), a projection scheme (holographic scheme), or thelike.

The audio output unit 152 may output audio data received from thewireless communication unit 110 or stored in the memory 170 in a callsignal reception mode, a call mode, a record mode, a voice recognitionmode, a broadcast reception mode, and the like. Also, the audio outputunit 152 may also provide audible output signals related to a particularfunction (e.g., a call signal reception sound, a message receptionsound, etc.) performed by the terminal 100. The audio output unit 152may include a receiver, a speaker, a buzzer or the like.

A haptic module 153 may generate various tactile effects that can befelt by a user. A representative example of tactile effect generated bythe haptic module 153 may be vibration. The intensity, pattern and thelike of vibration generated by the haptic module 153 may be controlledby a user's selection or the settings of the controller. For example,the haptic module 153 may output different vibrations in a combinedmanner or in a sequential manner.

Besides vibration, the haptic module 153 may generate various othertactile effects, including an effect by stimulation such as a pinarrangement vertically moving with respect to a contact skin, a sprayforce or suction force of air through a jet orifice or a suctionopening, a touch on the skin, a contact of an electrode, electrostaticforce, etc., an effect by reproducing the sense of cold and warmth usingan element that can absorb or generate heat, and the like.

The haptic module 153 may be implemented to allow the user to feel atactile effect through a muscle sensation such as the user's fingers orarm, as well as transferring the tactile effect through a directcontact. Two or more haptic modules 153 may be provided according to theconfiguration of the terminal 100.

An optical output unit 154 may output a signal for indicating an eventgeneration using light of a light source. Examples of events generatedin the terminal 100 may include a message reception, a call signalreception, a missed call, an alarm, a schedule notice, an emailreception, an information reception through an application, and thelike. A signal output by the optical output unit 154 may be implementedin such a manner that the terminal emits monochromatic light or lightwith a plurality of colors. The signal output may be terminated as theterminal senses a user's event checking.

The interface unit 160 may serve as an interface with every externaldevice connected with the terminal 100. For example, the interface unit160 may receive data transmitted from an external device, receive powerto transfer to each element within the terminal 100, or transmitinternal data of the terminal 100 to an external device. For example,the interface unit 160 may include wired or wireless headset ports,external power supply ports, wired or wireless data ports, memory cardports, ports for connecting a device having an identification module,audio input/output (I/O) ports, video I/O ports, earphone ports, or thelike.

The identification module may be a chip that stores various informationfor authenticating authority of using the terminal 100 and may include auser identity module (UIM), a subscriber identity module (SIM), auniversal subscriber identity module (USIM), and the like. In addition,the device having the identification module (referred to as ‘identifyingdevice’, hereinafter) may take the form of a smart card. Accordingly,the identifying device may be connected with the terminal 100 via theinterface unit 160.

Furthermore, when the terminal 100 is connected with an external cradle,the interface unit 160 may serve as a passage to allow power from thecradle to be supplied to the terminal 100 therethrough or may serve as apassage to allow various command signals input by the user from thecradle to be transferred to the terminal therethrough. Various commandsignals or power input from the cradle may operate as signals forrecognizing that the terminal is properly mounted on the cradle.

The memory 170 may store programs for operations of the controller 180and temporarily store input/output data (for example, phonebook,messages, still images, videos, etc.). The memory 170 may store datarelated to various patterns of vibrations and audio which are output inresponse to touch inputs on the touch screen.

The memory 170 may include at least one type of storage medium includinga Flash memory, a hard disk, a multimedia card micro type, a card-typememory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), aStatic Random Access Memory (SRAM), a Read-Only Memory (ROM), anElectrically Erasable Programmable Read-Only Memory (EEPROM), aProgrammable Read-Only memory (PROM), a magnetic memory, a magneticdisk, and an optical disk. Also, the terminal 100 may be operated inrelation to a web storage device that performs the storage function ofthe memory 170 over the Internet or other network.

As aforementioned, the controller 180 may typically control the generaloperations of the terminal 100. For example, the controller 180 may setor release a lock state for restricting a user from inputting a controlcommand with respect to applications when a state of the terminal meetsa preset condition.

Furthermore, the controller 180 may also perform controlling andprocessing associated with voice calls, data communications, videocalls, and the like, or perform pattern recognition processing torecognize a handwriting input or a picture drawing input performed onthe touch screen as characters or images, respectively. In addition, thecontroller 180 may control one or a combination of those components inorder to implement various exemplary embodiments disclosed herein.

The power supply unit 190 may receive external power or internal powerand supply appropriate power required for operating respective elementsand components included in the terminal 100 under the control of thecontroller 180. The power supply unit 190 may include a battery. Thebattery may be an embedded battery which is rechargeable or bedetachably coupled to the terminal body for charging.

Furthermore, the power supply unit 190 may include a connection port.The connection port may be configured as one example of the interfaceunit 160 to which an external (re)charger for supplying power torecharge the battery is electrically connected.

As another example, the power supply unit 190 may be configured torecharge the battery in a wireless manner without use of the connectionport. Here, the power supply unit 190 may receive power, transferredfrom an external wireless power transmitter, using at least one of aninductive coupling method which is based on magnetic induction or amagnetic resonance coupling method which is based on electromagneticresonance.

Various embodiments described herein may be implemented in acomputer-readable or its similar medium using, for example, software,hardware, or any combination thereof. For a hardware implementation, theembodiments described herein may be implemented within one or moreapplication specific integrated circuits (ASICs), digital signalprocessors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field programmable gate arrays(FPGAs), processors, controllers, micro-controllers, microprocessors,other electronic units designed to perform the functions describedherein, or a selective combination thereof. In some cases, suchembodiments are implemented by controller 180.

For a software implementation, certain embodiments described herein maybe implemented with separate software modules, such as procedures andfunctions, each of which perform one or more of the functions andoperations described herein. The software codes can be implemented witha software application written in any suitable programming language andmay be stored in memory (for example, memory 170), and executed by acontroller or processor (for example, controller 180).

The display unit 151 may output information processed in the terminal100. For example, the display unit 151 may display execution screeninformation of an application program driven in the terminal 100 or userinterface (UI) and graphic user interface (GUI) information in responseto the execution screen information.

The display unit 151 may include at least one of a liquid crystaldisplay (LCD), a thin film transistor-liquid crystal display (TFT-LCD),an organic light emitting diode (OLED), a flexible display, a3-dimensional (3D) display, and an e-ink display.

The display unit 151 may be implemented using two or more displays. Forinstance, a plurality of the display units 151 may be arranged on onesurface to be spaced apart from or integrated with each other, or may bearranged on different surfaces.

The display unit 151 may include a touch sensor which senses a touchonto the display unit so as to receive a control command in a touchingmanner. When a touch is input to the display unit 151, the touch sensormay be configured to sense this touch and the controller 180 maygenerate a control command corresponding to the touch. The content whichis input in the touching manner may be a text or numerical value, or amenu item which can be indicated or designated in various modes.

The touch sensor may be configured in a form of film having a touchpattern. Alternatively, the touch sensor may be integrally formed withthe display. For example, the touch sensor may be disposed on asubstrate of the display or within the display. The display unit 151 mayform a touch screen together with the touch sensor. Here, the touchscreen may serve as the user input unit 123.

The audio output module 152 may be implemented in the form of a receiverfor transferring voice sounds to the user's ear or a loud speaker foroutputting various alarm sounds or multimedia reproduction sounds.

The camera 121 may process video frames such as still or moving imagesobtained by the image sensor in a video call mode or a capture mode. Theprocessed video frames may be displayed on the display unit 151 orstored in the memory 170.

The user input unit 123 may be manipulated by a user to input a commandfor controlling the operation of the terminal 100.

The terminal 100 may also include a finger scan sensor which scans auser's fingerprint. The controller may use fingerprint informationsensed by the finger scan sensor as an authentication means. The fingerscan sensor may be installed in the display unit 151 or the user inputunit 123.

The microphone 122 may be formed to receive the user's voice, othersounds, and the like. The microphone 122 may be provided at a pluralityof places, and configured to receive stereo sounds.

The interface unit 160 may serve as a path allowing the terminal 100 toexchange data with external devices. For example, the interface unit 160may be at least one of a connection terminal for connecting to anotherdevice (for example, an earphone, an external speaker, or the like), aport for near field communication (for example, an Infrared DataAssociation (IrDA) port, a Bluetooth port, a wireless LAN port, and thelike), or a power supply terminal for supplying power to the terminal100. The interface unit 160 may be implemented in the form of a socketfor accommodating an external card, such as Subscriber IdentificationModule (SIM), User Identity Module (UIM), or a memory card forinformation storage.

At least one antenna for wireless communication may be disposed on theterminal body. A power supply unit 190 for supplying power to theterminal 100 may be disposed on the terminal body. The power supply unit190 may include a battery 191 which is mounted in the terminal body ordetachably coupled to an outside of the terminal body.

The battery 191 may receive power via a power source cable connected tothe interface unit 160. Also, the battery 191 may be (re)chargeable in awireless manner using a wireless charger. The wireless charging may beimplemented by magnetic induction or electromagnetic resonance.

FIG. 2 is a flowchart illustrating a method of controlling a specificspace for providing an environment optimized for a user. FIG. 3illustrates the control method of FIG. 2.

First, referring to FIG. 2, an electronic device according to anembodiment may learn specific information related to a user located in aspecific space through the learning data unit 130 (S210). Again,reference to learning data unit 130 is an example, and such teachingsapply as well to functions and the like performed by controller 180.

The specific space may be a three-dimensional space in which a person orobject can be located (or otherwise exist). For example, the specificspace may be an inner space of a vehicle, an inner space of a house, anoffice in a building, a conference room, and the like.

The learning data unit 130 may recognize (or sense) the user locatedwithin the specific space. Here, the operation of recognizing the userincludes an operation of sensing the user located within the specificspace, and an operation of inferring, guessing or predicting presence oridentity of a user who is expected to be located within the specificspace.

The learning data unit 130 may recognize the user located within thespecific space on the basis of at least one of information received froman external device, and sensing information sensed (detected) by thesensing unit 140 provided in the electronic device 100.

For example, the learning data unit 130 may recognize that the user islocated within the specific space based on an input signal received froma smart phone belonging to the user. As another example, the learningdata unit 130 may recognize the user located within the specific spacebased on an image captured by the camera 121 which is set to captureobject present within the specific space. As another example, thelearning data unit 130 may infer that a specific person will be locatedwithin the specific space based on analysis information obtained byanalyzing message contents exchanged with the specific person.

The learning data unit 130 may learn characteristic information relatedto the user located in the specific space. The characteristicinformation related to the user may include at least one of user-relatedbiometric information, user behavior information, log informationindicating a usage history of the space, and companion information usingthe space together.

In detail, the user's biometric information may include informationrelated to the user's body, such as heart rate information, blood flowinformation, height, body type, gender, and age. The user's behaviorinformation may include motion information of the user within the space,such as the user's clothing, voice information uttered by the user, theuser's moving line, and the user's movement. The log information mayinclude driving (or operation) information related to at least onedevice installed (or located) in the space, time information related tothe space being used by the user, weather information at the time whenthe user uses the space, and the like. The companion information may beinformation of a person located in the specific space at the same timeas the user.

The user characteristic information may be sensed through the sensingunit 140 provided in the electronic device, or may be collected fromdata input or output by the user to or from the electronic device. Forexample, the learning data unit 130 may sense biometric information suchas height and weight of the user located in the space and behaviorinformation regarding the user in the space, on the basis of imageinformation captured by the camera 121. As another example, the learningdata unit 130 may collect the log information related to the spaceindicating the usage history of the space, based on data that the userinputs or outputs to or from the electronic device.

In addition, the user characteristic information may be received from anexternal device capable of performing communication with the electronicdevice. External devices that may be implemented include devices capableof performing communication with electronic devices, and examples of theexternal devices may include a smart phone, a wearable device (e.g., asmart watch), a cloud server, and the like. For example, the learningdata unit 130 may perform communication with a smart watch worn by theuser to receive user-related heart rate information, user-related bloodflow rate information, and the like which is sensed by the smart watch.

The learning data unit 130 may learn the user characteristic informationbased on the machine learning technique. Specifically, the learning dataunit 130 may extract common elements from the user characteristicinformation, assign different weights to the common elements which arecommon to each of pieces of information, and learn the usercharacteristic information in a form of an artificial neural network.Here, the common element may be generated by learning information storedin the electronic device and information stored in thecommunication-available cloud server.

For example, as illustrated in FIG. 3, when a user located in a specificspace 300 drives a projector and a microphone together, the learningdata unit 130 may extract a common element that the projector and themicrophone are devices used for a conference, and assign a weight to anelement indicating a conference every time when the projector and themicrophone are driven.

As another example, when the user located in the specific space controlslighting at intermediate brightness and reproduces a sound source of acalm genre, the learning data unit 130 may extract a common element thatthe control of the lighting at the intermediate brightness and thereproduction of the sound source of the calm genre are used to relievestress, and assign a weight to an element indicating a relaxation (rest)even time when the lighting control and the sound source reproductionare executed.

Also, the learning data unit 130 may extract difference elements of theplurality of characteristic information in the same manner. In addition,the learning data unit 130 may learn the user characteristic informationbased on various statistical algorithms. Such learning methods arealready known and thus will not be described in detail herein.

The learning data unit 130 may learn the user characteristic informationand predict a use purpose of the space (i.e., space use purpose) on thebasis of the characteristic information (S220).

The user located in the specific space may have the space use purposefor using the specific space. The space use purpose refers to thepurpose that the user uses the specific space. For example, the user mayhave a conference as the space use purpose for a conference room. Also,the user may have relaxation as the space use purpose for a restingroom.

The space use purpose may be different for each user. For example, forthe same space, a user A may use the space for the purpose ofrelaxation, and a user B may use the space for the purpose of aconference.

When the user is recognized in the specific space, the learning dataunit 130 may predict (or decide) the space use purpose based on thecharacteristic information regarding the user located in the specificspace.

Specifically, the learning data unit 130 may decide the use purposehaving the highest probability among a plurality of use purposes thatthe user can use the space, based on the user characteristicinformation. This probability may be calculated using the learned usercharacteristic information based on the machine learning techniquedescribed in step S210.

For example, as illustrated in FIG. 3, when a person A is located in thespecific space, the learning data unit 130 may predict a conferencehaving the highest probability among a plurality of space use purposesas the space use purpose of the person A, on the basis of the learneduser characteristic information. As another example, when a person B islocated in the specific space, the learning data unit 130 may predictrelaxation having the highest probability among a plurality of space usepurposes as the space use purpose of the person B, on the basis of thelearned user characteristic information.

Therefore, even if the user does not make a separate action within thespecific space or does not input a control command for controlling thespace, the learning data unit 130 may predict the space use purpose forwhich the user is expected to use the space, on the basis of the usercharacteristic information learned through the machine learningtechnology.

Also, the space use purpose may be decided by the user recognized withinthe space and a time at which the user is located within the space. Forexample, when a person A is located in the specific space at 1:00 pm,the learning data unit 130 may predict a conference as the space usepurpose. On the other hand, when the person A is located in the specificspace at 6:00 pm, the learning data unit 130 may predict dinner as thespace use purpose.

Meanwhile, at least one device may be installed in the specific spaceaccording to a space use purpose. For example, a projector, amicrophone, a speaker, a lighting, a chair, a desk, and the like may beprovided in a conference room. As another example, a sofa, a TV, aspeaker, a lighting, an air conditioner, a clock, and the like may beprovided in a living room. In addition, various electronic devices maybe installed in the specific space.

The learning data unit 130 may set a driving condition of at least onedevice installed in the specific space to meet a predicted use purposewhen a user located in the specific space is sensed. That is, thelearning data unit 130 may set a driving condition for driving at leastone device installed in the space by a driving method according to thespace use purpose, in order to allow the user located in the space touse the space more conveniently according to the predicted use purpose.

For example, as illustrated in FIG. 3, when a user located in thespecific space 300 is detected, the learning data unit 130 may predictthe user's space use purpose as ‘conference’. The learning data unit 130may set driving conditions associated with a projector, a lighting, anda microphone to meet the ‘conference’ as the space use purpose. Forexample, the learning data unit 130 may set driving conditions such as‘Projector ON’, ‘Lighting OFF near the projector//Lighting ON away fromthe projector, and ‘Microphone ON’.

The space use purpose may include a preset driving condition for atleast one device. The driving condition for the at least one device maybe extracted from the user characteristic information.

For example, the learning data unit 130 may extract operationinformation related to power-on of the project, power-on of themicrophone, and the lighting control from log information related to thespace used for the purpose of a conference. The learning data unit 130may set a driving condition based on operation information related to atleast one device extracted from the log information and store the setdriving condition in the memory 170.

In this instance, the learning data unit 130 may set the drivingcondition of the at least one device by considering not only the loginformation regarding the space used by the user located in the specificspace but also log information regarding the space used by other usersfor the same space use purpose as the user. Here, the log informationregarding the space used by the other users may be stored in acommunication-available external server, or may be stored in the memoryof the electronic device.

That is, one embodiment can provide a space environment according to amethod that the user located within the specific space uses the specificspace, and also create an environment of the specific space to beoptimized for a specific use purpose by considering even driving methodsof other users who have used the specific space for the same space usepurpose. This may result in improving user convenience in using thespecific space.

In addition, the learning data unit 130 may set a driving condition ofat least one device in consideration of the space use purpose andbiometric information related to the user located in the specific space.For example, when the space use purpose is a conference, the learningdata unit 130 may arrange a seat or chair closest to the projector inconsideration of information related to the user's height or eyesight atthe time of arranging the user's seat in a conference room. As anotherexample, when the space use purpose is the conference, the learning dataunit 130 may arrange the user's seat at the farthest position from anair conditioner in consideration of the user's body temperature at thetime of arranging the user's seat in the conference room.

The learning data unit 130 may transfer a driving command according tothe driving condition of the at least one device to the controller 180so that the at least one device can be driven according to the usepurpose. The controller 180 may drive the at least one device based onthe transferred driving command. For example, the controller 180 maycontrol a lighting installed in the specific space to be turned on.

As such, the foregoing description presents a method of predicting theuse purpose of the specific space based on the characteristicinformation regarding the user located in the specific space andcontrolling the specific space to meet the predicted use purpose. Thismay result in improving the user convenience in using the specificspace.

FIG. 4 is a flowchart illustrating a method of controlling a specificspace when a plurality of users are located in the specific space. FIGS.5A and 5B illustrate the control method of FIG. 4.

The electronic device according to an embodiment may control anoperation of at least one device located in a specific space accordingto a user located in the specific space. The specific space may be usedby plural users. In this scenario, a use purpose for the specific spacemay differ according to what kind of users are using the specific space.Therefore, discussion will now include a method of creating anenvironment of a space when a plurality of users are using the space.

Referring to FIG. 4, the learning data unit 130 of the electronic device100 may learn characteristic information related to each of theplurality of users located within the specific space (S410).

First, the learning data unit 130 may recognize (or detect) theplurality of users located within the specific space. In detail, thelearning data unit 130 may detect the plurality of users located withinthe specific space through the sensing unit which is capable ofdetecting an object located within the specific space. For example, thelearning data unit 130 may detect the plurality of users through acamera positioned within the specific space. In this embodiment, thelearning data unit 130 may analyze image information in which theplurality of user are captured, and identify the plurality of usersbased on the analysis result.

The learning data unit 130 may learn characteristic information relatedto each of the plurality of users when the plurality of users areidentified. This learned information may be generated as a personalizeddatabase corresponding to each of the plurality of users. Descriptionrelated to the learning of the learning data unit is similar to that ofblock S210 of FIG. 2, and is not repeated here.

The learning data unit 130 may predict a use purpose of the specificspace based on a combination of the characteristic information relatedto the plurality of users (S420).

The learning data unit 130 may combine the characteristic informationrelated to each of the plurality of users when the plurality of userslocated within the specific space are detected. In detail, the learningdata unit 130 may extract a common element from the characteristicinformation related to the plurality of users. For example, when anoperation ‘Projector ON’ is commonly included in characteristicinformation related to a person A and characteristic information relatedto a person B, the learning data unit 130 may extract the operation‘Projector ON’ as the common element. As another example, when anoperation ‘Music reproduction’ is commonly included in thecharacteristic information related to person A and the characteristicinformation related to person B, the learning data unit 130 may extractthe operation ‘Music reproduction’ as the common element.

Also, the learning data unit 130 may extract as the common elementinformation generated among the plurality of users, such asconversations among the plurality of users, behavior informationregarding the plurality of users, and the like, from each characteristicinformation. For example, the learning data unit 130 may extract aconversation between the persons A and B as the common element.

The learning data unit 130 may predict (or decide) a use purpose thatthe plurality of users use the specific space, on the basis of theextracted common element. For example, as illustrated in FIG. 5A, thelearning data unit 130 may predict ‘conference’ as the use purpose,based on the operation ‘Projector ON’ and a conversation between thepersons A and B of, for example, “The conference will begin”. As anotherexample, the learning data unit 130 may predict ‘date’ as the usepurpose, based on the operation ‘Music reproduction’ and a conversationbetween persons A and C of, for example, ‘Go out with me’.

That is, this feature may recognize the change of the purpose for usingthe specific space according to a group (or combination) of the userslocated within the specific space.

For ease of discussion, the above description is based on the commonelement of the plurality of characteristic information, but a differenceelement of the plurality of characteristic information may also beextracted in a similar manner. That is, the learning data unit 130 mayextract the common element and the difference element of the pluralityof characteristic information, and predict the use purpose by combininginformation related to the common element and the difference element.

Referring back to FIG. 4, the controller 180 of the electronic device100 according to the present invention may control at least one devicelocated in the specific space (S430). The learning data unit 130 may seta driving condition of the at least one device in the specific spacebased on the predicted use purpose. For example, the learning data unit130 may set the driving condition of the at least one device to‘conference’ when ‘conference’ is predicted as the use purpose. Themethod of setting the driving condition of the at least one device hasbeen described in relation to block S230, so further discussion isomitted.

The learning data unit 130 may then transfer the driving condition ofthe at least one device to the controller 180 such that the at least onedevice can be driven according to the set driving condition. Thecontroller 180 may thus control the at least one device based on thetransferred driving condition of the at least one device.

For example, referring to FIG. 5A, when ‘conference’ is the use purpose,the controller 180 may execute a seat positioning control that the seatsare into postures to be appropriate for the users to be seated,projector ON, microphone ON, lighting OFF, and an air-conditioningsystem control for a temperature/humidity control.

As another example, as illustrated in FIG. 5B, when ‘date’ is the usepurpose, the controller 180 may execute a seat positioning control thatthe seats are into postures to be appropriate for the users to beseated, speaker ON for a music reproduction, a lighting brightnesscontrol and an air-conditioning system control for atemperature/humidity control.

Description has been provide of methods of providing the specific spaceaccording to an environment appropriate for the plurality of users whenthe plurality of users located within the specific space are detected.Accordingly, such features provide the most appropriate or desirableenvironment to the users located within the specific space.

Reference is now made to FIGS. 6A(a), 6A(b), 6A(c), 6A(d), 6B(a), 6B(b),6C that illustrate a method of recognizing a plurality of users who usea specific space.

The learning data unit 130 of the electronic device 100 may detect aplurality of users currently located within a specific space, or predicta plurality of users expected to be located within the specific space infuture. In this example, learning data unit 130 may receive informationrelated to each of the plurality of users from an external device, orreceive image information in which the plurality of users are capturedthrough the camera provided in the electronic device 100. Theinformation related to each of the plurality of users may includeinformation related to conversations associated with the specific spaceamong the plurality of users, conversation information, informationrelated to a post uploaded on an SNS server and the like.

For example, as illustrated in FIG. 6A(a), the learning data unit 130may analyze conversations between persons A and B. The conversationsbetween the persons A and B may be received from an external device(mobile terminals belonging to A and B), or extracted from a call signalevent generated in the electronic device 100.

The learning data unit 130 may analyze the conversation based on apreset algorithm (e.g., a conversation analysis algorithm). The learningdata unit 130 may extract information indicating that persons A and Bare to be located in the specific space at a specific time, based on theanalyzed result. That is, the learning data unit 130 may predict thatthe persons A and B will be located in the specific space at thespecific time. An algorithm well known in the related art may be used asthe conversation analysis algorithm, so description thereof will beomitted.

In this instance, the learning data unit 130 may predict (or decide) ause purpose of the persons A and B for the specific space by combiningcharacteristic information related to each of the persons A and B andthe conversation. For example, as illustrated in FIG. 6A(b), thelearning data unit 130 may predict ‘conference’ as the use purpose.

As illustrated in FIG. 6A(d), the learning data unit 130 may transferthe driving condition for controlling the device installed in thespecific space to the controller 180, in order to control the specificspace. The controller 180 may control the device based on the drivingcondition transferred from the learning data unit 130. Therefore, theuser can be provided with an appropriate environment according to thespace use purpose.

Additionally, the learning data unit 130 may transfer notificationinformation related to the specific space to each of the plurality ofusers to guide the predicted plural users to use the specific space. Forexample, as illustrated in FIG. 6A(c), the learning data unit 130 maytransmit notification information including schedule information 610, alocation 620 of the specific space and a document 630 associated withthe use purpose, to contact information (phone number) corresponding toeach of the persons A and B. Accordingly, the plurality of users mayconveniently check various information related to the specific space.The notification information may be output in at least one of visible,audible and tactile manners.

Also, the controller 180 may decide an output time point of thenotification information according to the schedule information relatedto the specific space. For example, the controller 180 may output thenotification information 1 hour before a specific time which is includedthe schedule information related to the specific space. Therefore, theusers can check the notification information at appropriate time points.

Alternatively, the learning data unit 130 may transmit the notificationinformation at different time zones, considering a location of each ofthe plurality of users and the location information related to thespecific space. For example, if a user is located one hour away from thespecific space, the notification information may be transmitted one hourbefore, and if the user is located five minutes away from the specificspace, the notification information may be transmitted five minutesbefore. In other words, the learning data unit 130 may provideappropriate information for each user by transmitting the notificationinformation at an appropriate time point in consideration of the currentlocation of each user.

Also, as illustrated in FIG. 6B(a), the learning data unit 130 mayrecognize (or detect) a plurality of users located in a specific spacethrough a camera that is installed in the specific space to capture theinside and outside of the specific space. In this instance, the learningdata unit 130 may recognize the users located in the specific spacethrough an image analysis algorithm, combine characteristic informationrelated to the recognized users, and predict a space use purpose (e.g.,a conference such as that shown in FIG. 6B(b)).

In addition, the learning data unit 130 may sense that a use purpose fora specific space has changed. More specifically, the learning data unit130 may detect that an additional user is located in a specific space ora user located in the specific space has left. For example, asillustrated in FIG. 6C, the learning data unit 130 may further detect auser C in a state where users A and B are already located in thespecific space.

In this instance, the learning data unit 130 may predict the space usepurpose again, based on a combination of characteristic informationrelated to the additional user and characteristic information related toexisting users. If the space use purpose has not changed, the learningdata unit 130 may not generate a separate control command. On the otherhand, if the space use purpose is changed, the learning data unit 130may set a driving condition of at least one device to meet the changeduse purpose. For example, as illustrated in FIG. 6C, the learning dataunit 130 may change the space use purpose from ‘conference’ to‘relaxation’. In this instance, the learning data unit 130 may perform‘Lighting ON’, ‘Projector OFF’, and ‘Seat posture control’ to meet thenew relaxation purpose.

Alternatively, the learning data unit 130 may predict that timeinformation scheduled for a plurality of users to be located in thespecific space is changed. In this instance, the learning data unit 130may transmit notification information so that the users recognize thechanged time information. Therefore, the users can recognize the changeduse time.

Alternatively, when only a portion of the users is located in thespecific space, the learning data unit 130 may transmit notificationinformation to the remaining users. Also, current location informationrelated to the remaining users may be transmitted to those several userslocated in the specific space. That is, when a plurality of users to belocated in the specific space are recognized, the learning data unit 130may provide each user with notification information related to thespecific space, to faithfully play the role of an individual secretaryof each user.

The foregoing description is an example of a method of recognizing userslocated in the specific space. Description will be provided of a methodof generating schedule information related to a specific space by acombination of schedule information related to each of a plurality ofusers. FIGS. 7A, 7B illustrate a method of generating scheduleinformation related to a specific space by combining scheduleinformation regarding a plurality of users.

Referring first to FIG. 7A, the learning data unit 130 may receiveschedule information related to each of a plurality of users A, B, C andD through communication with a plurality of external devices. Forexample, the learning data unit 130 may receive schedule informationstored in each of the plurality of external devices through Bluetoothcommunication. The schedule information may be information includingplace information, time information and date information.

The learning data unit 130 may learn the schedule information related toeach of the plurality of users based on machine learning technology. Indetail, the learning data unit 130 may extract a common element and adifference element of each of those pieces of schedule information, andgenerate schedule information related to the specific space based on theextracted common element and difference element.

For example, as illustrated in FIG. 7B, the learning data unit 130 maygenerate schedule information indicating that the users A, B, C and Dare to use a second conference room at 7:00 pm on Mar. 1, 2017. That is,the learning data unit 130 may also predict a user to use the specificspace, a time to use the specific space, and a date to use the specificspace.

In this instance, the learning data unit 130 may control at least onedevice located at the specific space according to the generated scheduleinformation related to the specific space. As such, the learning dataunit 130 may set a driving condition of the device based on the scheduleinformation and characteristic information related to each of the usersusing the specific space. These feature provides, among other things, anoptimized environment of the specific space according to the schedulesof the users.

When the schedule information related to the specific space isgenerated, the learning data unit 130 may transmit notificationinformation notifying the schedule information related to the specificspace to an external device corresponding to at least one user includedin the schedule information related to the specific space. For example,when the users A, B, C and D are included in the schedule informationrelated to the specific space, the controller 180 may transmitnotification information notifying the schedule information related tothe specific space to smart phones belonging to the users A, B, C and D.The notification information may include place information, a movingpath, and the like.

Meanwhile, although not illustrated, the learning data unit 130 maydecide a location of a specific space in which a plurality of users areto gather, on the basis of schedule information related to the users. Indetail, the learning data unit 130 may decide information related to onespace to be set as the specific space among at least one place that theusers usually frequently visit. That is, a space to be used as thespecific space can be decided. In this example, the learning data unit130 may provide an optimal environment by controlling at least onedevice located at the decided specific space through communication.

The foregoing description is an example of a method of generating theschedule information related to the specific space using the scheduleinformation. Accordingly, optimal meeting place and meeting time may beidentified by considering schedules of all of users through artificialintelligence. The system may set a driving condition of at least onedevice located at a specific space based on characteristic informationrelated to users located within the specific space, and control the atleast one device to be driven according to the set driving condition.Therefore, an optimized space environment may be realized according tospace use characteristics of the users.

A further feature is that at least one device installed in a specificspace can be driven to meet a space use purpose of a user located in thespecific space, thereby improving the user convenience in using thespace.

Next described is a method in which an electronic device executes avehicle control according to an embodiment of the present invention. Inparticular, FIG. 8 is a flowchart illustrating a method of controlling avehicle using an electronic device and FIGS. 9 to 12 illustrate acontrol method of FIG. 8.

The electronic device may execute communication with a vehicle in awired or wireless manner, and control the vehicle through thecommunication with the vehicle. The communication may be short-rangecommunication, Vehicle-to-everything (V2X) communication, opticalcommunication, and the like, applied in the vehicle. Other communicationmethods to be employed in the vehicle may also be used. When theelectronic device is used for the vehicle control, it may also bereferred to as a driving assistant device, a vehicle control device, avehicle driving device and the like. A method of controlling a vehicleusing an electronic device will now be described in more detail.

Referring to FIG. 8, the learning data unit 130 of the electronic devicemay learn boarding status information (S810).

Referring to FIG. 9, the boarding status information may include one ormore of biometric information related to a passenger seated in avehicle, information related to a surrounding environment of thepassenger at the time of boarding the vehicle, or information related toa vehicle control at the time when the passenger boards the vehicle. Thebiometric information related to the passenger may include one or moreof heart rate information, body temperature information and the like.The information related to the surrounding environment may includeboarding time information, passenger voice information, boarding seatinformation, weather information at the time of boarding, informationrelated to an air conditioning system such as temperature informationand humidity information around the passenger, speaker volumeinformation, display ON/OFF information, seat arrangement information,companion information, and the like. The information related to thevehicle control may include destination information related to thepassenger, route (moving path) information related to the vehicle,average speed information related to the vehicle, driving styleinformation, and the like.

The learning data unit 130 may learn the boarding status informationbased on the machine learning technique. Specifically, the learning dataunit 130 may analyze the learned boarding status information based on aplurality of elements. Here, the plurality of elements may be commonelements and difference elements extracted from a plurality of boardingstatus information.

Referring now to FIG. 10, the vehicle may include air conditioningsystems 1010 a, 1010 b, 1010 c and 1010 d, speakers 1020 a, 1020 b, 1020c and 1020 d, and display units 1030 a, 1030 b, 1030 c, 1030 d for seats1000 a, 1000 b, 1000 c and 1000 d, respectively, disposed in thevehicle. The air conditioning system, the speaker, and the display unitprovided for each seat of the vehicle may be independently controlled.Therefore, the passenger seated in each seat can be provided with anoptimized environment according to the tendency or desire of thepassenger.

The learning data unit 130 may learn boarding status information foreach seat occupied by the passenger. For example, when a plurality ofpassengers board the vehicle, the learning data unit 130 may learnboarding status information for each seat occupied by the passengers.Therefore, the learning data unit 130 may learn locations of boardingseats according to the combination of the plurality of passengers andinformation related to a surrounding environment of each passengerseated in the seat.

The learning data unit 130 may predict a boarding event based on thelearned information (FIG. 8, S820). The boarding event may be an eventthat a passenger gets in the vehicle at a specific time.

The learning data unit 130 may predict a boarding event that a specificperson is to board the vehicle at a specific time when it is detectedthat the specific person boards the vehicle at a specific time for apreset number of times or more. For example, the boarding event may bean event where Mom boards the vehicle at 7:00 pm.

The learning data unit 130 may predict a boarding purpose of theboarding event based on a machine learning technique. The boardingpurpose is a purpose that a passenger wants to use the vehicle, forexample, a purpose of going to work, a purpose of travel, or a purposeof shopping. In addition, there may be various purposes that thepassengers want to do after boarding the vehicle.

A method of predicting the boarding purpose will now be described inmore detail. The learning data unit 130 may classify boarding statusinformation related to passengers stored in an external server andboarding status information related to the passengers boarded in thevehicle into preset reference elements, on the basis of a machinelearning technique. The preset reference elements may be common elementsand difference elements included in those pieces of the boarding statusinformation.

The learning data unit 130 may set a weight for each of the referenceelements, and predict a specific boarding purpose according to passengerinformation and boarding time information included in the boardingevent. For example, if it is predicted through the boarding event thatthe passenger information is ‘Mom’ and the boarding time is ‘7:00 pm’,the learning data unit 130 may predict the boarding purpose as ‘Homefrom work’ based on the boarding status information related to the Momas the passenger. As another example, if it is predicted through theboarding event that the passenger information is ‘Mom’ and the boardingtime is ‘10:00 am’, the learning data unit 130 may predict the boardingpurpose as ‘shopping’ based on the passenger status information relatedto the Mom as the passenger. That is, the learning data unit 130 maypredict the most proper boarding purpose by considering all of thepassenger information, the boarding time information, and the boardingstatus information related to the passenger.

On the other hand, the learning data unit 130 may predict a boardingevent including a plurality of passengers. In this instance, thelearning data unit 130 may predict a boarding purpose based on acombination of boarding status information of each of the plurality ofpassengers. In this regards, reference is made to FIG. 12A, whichdepicts various boarded passenger configurations in a vehicle, FIG. 12B,which is a table of boarding events data for each of the configurationsof FIG. 12A, and FIG. 12C, which is a table of vehicle control data foreach of the configurations of FIG. 12A.

In these figures, when ‘Dad’, ‘Mom’, and ‘Kid’ board the vehicletogether, the learning data unit 130 may predict the boarding purpose asa travel, on the basis of a combination of boarding status informationrelated to each of those passengers. As another example, when ‘Mom’ and‘Kid’ board the vehicle, the learning data unit 130 may predict theboarding purpose as ‘Home from school’ by combining boarding statusinformation related to each of those passengers. As another example,when ‘Mom’ boards the vehicle alone, the learning data unit 130 maypredict the boarding purpose as ‘shopping’.

The combination of the boarding status information will be described inmore detail as the learning data unit 130 may extract common elementsand difference elements from the boarding status information related toeach of a plurality of passengers, and assign different weights to therespective elements. The learning data unit 130 may predict the boardingpurpose based on the elements with the different weights assigned.

Next, the learning data unit 130 may extract at least one controlinformation, which is to be executed in response to the predictedboarding event, from the learned information (FIG. 8, S830).

Referring now to FIG. 11, the learning data unit 130 may extract atleast one control information from the boarding status informationrelated to the passengers included in the predicted boarding event tomeet the boarding purpose.

The control information related to the vehicle may include one or moreof control information for executing a power train driving control, achassis driving control, a door/window driving control, a safety devicedriving control, a lamp driving control, an air conditioning drivingcontrol, a vehicle driving control, a parking-out control, a parking-incontrol, seat control, a user interface device control, and the like.The user interface device may include an input unit, a display unit, anaudio output unit and a haptic output unit, a camera, and a biometricsensing unit for receiving voice, gesture, touch, and mechanical input.

The learning data unit 130 may extract the vehicle control informationfrom the boarding status information of the passengers included in theboarding event to meet (match) the boarding purpose. For example, thelearning data unit 130 may extract at least one control information fromthe boarding status information related to ‘Mom’ when ‘Mom’ is includedin the boarding event. Therefore, as an example, the environment of thevehicle can be created according to the passenger.

On the other hand, when a plurality of passengers are included in thepredicted boarding event, the learning data unit 130 may extract controlinformation related to seats to be used by the passengers, from theboarding status information regarding the passengers.

For example, when two passengers board the vehicle, the learning dataunit 130 may predict based on the boarding status information related tothe passengers that one of the passengers is to be seated in a driverseat and the other passenger is to be seated in a passenger seat or arear seat. Then, the learning data unit 130 may extract controlinformation related to the driver seat from the boarding statusinformation related to the passenger expected to be seated in the driverseat, and obtain control information related to the rear seat from theboarding status information related to the passenger expected to beseated in the rear seat.

That is, the learning data unit 130 may perform a different control foreach seat using the boarding status information of the passengers to beseated in the respective seats. Therefore, the learning data unit 130can provide a vehicle environment suitable for the boarding purpose inthe vicinity of the seat occupied by the passenger.

As a further feature, the controller 180 may control the vehicle basedon the extracted at least one control information (FIG. 8, S840).

For instance, the learning data unit 130 may transmit the extractedcontrol information to the controller 180 so that the vehicleenvironment can be created according to the predicted boarding event.The controller 180 may thus control the vehicle based on the extractedcontrol information. The control information may be a signal or data forcontrolling the vehicle as described above.

Meanwhile, the learning data unit 130 may set control information forcontrolling the vehicle just before a boarding time included in theboarding event, so that the environment of the vehicle is created atboarding time. For example, the learning data unit 130 may includesetting information set in the control information such that the vehicleis controlled just before the boarding time. In this instance, thecontroller 180 may control the vehicle at a time point set in thecontrol information. As another example, the learning data unit 130 maytransmit the control information to the controller 180 just before theboarding time. As such, the controller 180 may promptly control thevehicle when the control information is received. Accordingly, aninternal environment of the vehicle before the passenger boards thevehicle is created.

Alternatively, the learning data unit 130 may transmit at least onecontrol information to the controller 180 when a passenger included inthe boarding event is detected through the camera 121. In this instance,vehicle environment most suitable for a passenger by controlling thevehicle at the time when the passenger actually boards the vehicle isprovided.

The foregoing example relates to a method of controlling the vehicleaccording to the boarding event in the electronic device. Next will bedescribed a method of controlling the vehicle according to the abovecontrol method, and reference will again be made to FIGS. 12A, 12B, and12C.

According to FIGS. 12A, 12B, 12C, the learning data unit 130 may predicta boarding event including that passengers are Dad, Mom and a kid, and aboarding time is 7:00 pm on Friday, and determine a boarding purpose ofthe predicted boarding event as shopping.

When the plurality of passengers are included in the predicted boardingevent, the learning data unit 130 may extract control informationrelated to seats occupied by the passengers, respectively, from boardstatus information of each passenger.

In more detail, when Dad occupies the driver seat, the learning dataunit 130 may extract control information related to the driver seat fromboard status information related to Dad. When Mom occupies a passengerseat, the learning data unit 130 may extract control information relatedto the passenger seat from boarding status information related Mom, andlikewise when the kid occupies a rear seat, the learning data unit 130may extract control information related to the rear seat from boardingstatus information related to the kid. Therefore, an environmentoptimized for each seated passenger may be achieved.

The learning data unit 130 may also extract at least one controlinformation from a combination of boarding status information related tothe plurality of passengers, in order to meet a boarding purpose. Forexample, the learning data unit 130 may extract control information,‘Trunk open’ and ‘Destination shopping center’ to match ‘Shopping’.Accordingly, the user is provided with a vehicle environment appropriatefor the user's vehicle use purpose, even without a user's having to makea separate action.

Referring still to FIGS. 12A, 12B, 12C, when a boarding event thatincludes the passengers are Mom and a kid and a boarding time is 6:00pm, then an everyday parameter is predicted and the learning data unit130 may extract different control information from that of thepreviously-described boarding event. That is, different controlinformation is extracted according to the passengers and the boardingtime included in the boarding event. Therefore, various boarding eventscan be predicted in order to provide vehicle environments which are themost appropriate for the predicted boarding events.

Various methods of predicting boarding events and controlling thevehicle according to the boarding events has been described. Furtherembodiments include a method of providing boarding seat information to aplurality of passengers included in a boarding event, as will now bedescribed.

FIGS. 13A(a), 13A(b), 13B, and 13C illustrate methods of providingpassenger seat information to passengers in accordance with oneembodiment of the present invention. When a boarding event including aplurality of passengers is predicted, the controller 180 may decideboarding seats to be seated by the plurality of passengers. For example,when a boarding event including persons A and B is generated, thecontroller 180 may set the boarding seats such that person A occupies adriver seat and person B occupies a passenger seat.

When the boarding seats of the plurality of passengers are decided, thecontroller 180 may provide boarding seat information to the plurality ofpassengers. The controller 180 may provide the boarding seat informationin at least one of visible, audible or tactile manners.

The controller 180 may also output the boarding seat information throughan external device capable of performing communication with theelectronic device, or through an interface of the vehicle provided withthe electronic device.

Referring to FIG. 13A(a), the controller 180 may transmit a messageincluding the boarding seat information to contact informationcorresponding to each of the persons A and B such that persons A and Bcan recognize their seat locations. For example, a mobile terminal ofthe person B may receive notification information 1310 “The weather iscold today. The passenger seat has been heated up.” Therefore, person Bcan recognize that the passenger seat which he or she is to be seatedhas been heated up. As another example, referring to FIG. 13A(b),notification information 1320 may include an image 1330 indicatinglocations to be seated by the passengers. Therefore, the passengers canrecognize the seat information through smart phones belonging to thepassengers, respectively.

As another example, as illustrated in FIG. 13B, the controller 180 mayoutput light through a light output unit (e.g., LED output unit)provided on a door handle of the vehicle corresponding to the decidedseat information. In this example, the light output unit provided oneach door handle may output light of a different color preset for eachpassenger. Therefore, the passengers can intuitively recognize thelocations to be seated according to the different colors of light.

As another example, as illustrated in FIG. 13C, the controller 180 mayvisually output the decided seat information on a display unit (e.g.,HUD) provided in the vehicle.

The method of transmitting the notification information illustrated inFIGS. 13B and 13C can be implemented when there is no contactinformation corresponding to each passenger.

A method of providing the boarding seat information has been described.Such information can guide the passengers to their respective seats withthe appropriate environment created for each respective passenger.

FIGS. 14A(a), 14A(b), 14A(c), 14B(a), 14B(b), 14B(c) illustrate methodsof predicting passengers included in a boarding event. In particular,the learning data unit 130 may receive message contents from an externaldevice and analyze these contents based on a conversation analysisalgorithm. The conversation analysis algorithm can use thepreviously-known algorithm, so detailed description thereof will beomitted.

Referring to FIG. 14A(a), the learning data unit 130 may predict aboarding event based on the analysis result. For example, as illustratedin FIG. 14A(b), the learning data unit 130 may predict the boardingevent indicating that passengers are a user (Me), and persons A and Band a boarding time is 2:10 pm.

The learning data unit 130 may generate boarding seat informationrelated to the plurality of passengers based on a combination ofboarding status information of each of the passengers. The boardingstatus information may include analysis information of the messagecontents. In this example, the learning data unit 130 may generate theboarding seat information related to the passengers by assigning ahigher weight to the latest information when the analysis information ofthe message contents includes the latest information.

For example, the learning data unit 130 may generate the boarding seatinformation indicating that the user is seated in a driver seat, personA is seated in a passenger seat and person B is seated in the rear seat,based on pre-learned boarding seat information. In this instance, whenbody-related information that B has a bad back is extracted from theanalysis information on the message contents, since the body-relatedinformation is the latest information, the learning data unit 130 maychange the generated boarding seat information by assigning a higherweight. That is, the learning data unit 130 may change the boarding seatinformation to indicate that the user is seated in the driver seat,person A is seated in the rear seat and person B is seated in thepassenger seat. Accordingly, optimal seat arrangement can be set byconsidering not only information related to seats that the passengershave been seated before but also a current health condition.

The learning data unit 130 may also detect passengers to be included ina boarding event based on surrounding image information of the vehicle.For example, as illustrated in FIG. 14B(a), the learning data unit 130may detect ‘Me’, ‘A’ and ‘B’ as passengers based on an image capturedthrough the camera. In this instance, as illustrated in FIG. 14B(b), thelearning data unit 130 may predict the boarding event that thepassengers are ‘Me’, ‘A’ and ‘B’ and a boarding time is now.

The learning data unit 130 may extract body-related information (e.g.,height, body type) regarding each passenger from the passenger-capturedimage. In this instance, the learning data unit 130 may generateboarding seat information by assigning a higher weight to thebody-related information extracted from the image. For example, thelearning data unit 130 may arrange ‘A’ who is big in the passenger seat,and ‘B’ who is small in the rear seat by considering the body-relatedinformation.

The learning data unit 130 may also detect a passenger, whose boardingstatus information is not provided, among passengers detected from apassengers-captured image. That is, the learning data unit 130 maydetect a passenger who has no previous history of boarding the vehiclefrom this image. In this example, the learning data unit 130 maygenerate boarding seat information by considering the body-relatedinformation, such as gender, age, height, body shape and the likeextracted from the image.

A method has been described in which passengers included in the boardingevent are recognized by the electronic device. Another embodiment isdepicted in FIGS. 15A, 15B, 15C, which illustrate a method ofreproducing content while driving a vehicle with a plurality ofpassengers seated.

The learning data unit 130 may reproduce a content through an audiosystem provided in the vehicle while the vehicle with the passengers isdriven. Here, the content may be a sound source, a video and the like.The learning data unit 130 may calculate content occupancy rates of thepassengers by comparing boarding status information of each of thepassengers with attribute information of the content currentlyreproduced in the vehicle. In detail, the learning data unit 130 may sethigh content occupancy rate to a passenger whose boarding statusinformation includes content reproduction tendency information which isthe same as the attribute information on the content currentlyreproduced in the vehicle. For example, when a sound source of a genre‘Love’ is currently reproduced, the learning data unit 130 may set theoccupancy rate of a passenger whose boarding status information includessound source information of the genre ‘love’ to be higher than those ofthe other passengers.

In this instance, the learning data unit 130 may provide a preferredsound source of a passenger with a high content occupancy rate, as arecommended sound source. For example, as illustrated in FIGS. 15A, 15B,when a content occupancy rate of a person A is higher than that of aperson B in a state in which the persons A and B have boarded thevehicle, the learning data unit 130 may provide A-preferred soundsources A-1 and A-2 as the recommended sound sources. The A-preferredsound sources may be the sound source information received from a mobileterminal belonging to the person A or may be a sound source extracted onan external server according to the tendency of person A.

Similarly, as illustrated in FIGS. 15A, 15C, the learning data unit 130may provide sound sources B-1 and B-2 as recommended sound sources whenthe content occupancy rate of the person B is higher than the contentoccupancy rate of the person A.

FIGS. 16A(a), 16A(b), 16B(a), 16B(b) illustrate methods of controllingan environment of a vehicle during driving of the vehicle. Inparticular, these figures depict methods of controlling an environmentof a vehicle according to boarding status information regarding a userwhile driving the vehicle

The learning data unit 130 may detect boarding status informationregarding a plurality of users (passengers) seated in the vehicle whilethe vehicle is driven. The learning data unit 130 may control asurrounding environment of a seat occupied by each passenger based onthe boarding status information regarding each passenger.

For example, as illustrated in FIG. 16A(a), the learning data unit 130may detect that a passenger seated in a rear seat is in a sleepingstate. In this instance, as illustrated in in FIG. 16A(b), the learningdata unit 130 may minimize a volume of a speaker so that a sound sourceis not output through a speaker arranged at the rear seat.

As another example, as illustrated in FIG. 16B(a), the learning dataunit 130 may detect that a passenger seated in the rear seat is in acall-conversation state (e.g., talking on the phone). In this instance,as illustrated in FIG. 16B(b), the learning data unit 130 may minimize avolume of a speaker arranged at the rear seat, and execute a noisecanceling function such that a call sound is not heard by otherpassengers seated in the driver seat and the passenger seat. Thus, anoptimal seat environment for each seat is provided based on the boardinginformation on the passengers while the vehicle is driven or isotherwise occupied.

The foregoing description has been given of the method of controllingthe environment of the vehicle according to the boarding statusinformation involved in the passengers while driving the vehicle.

FIGS. 17A(a), 17A(b), 17A(c), 17B(a), 17B(b), 17B(c) illustrateembodiments of controlling a vehicle when an error occurs in predictionof a boarding event. The learning data unit 130 may predict a boardingevent and control the vehicle according to the predicted boarding event.On the other hand, the learning data unit 130 may detect an occurrenceof another event different from the boarding event at a time point whenit is predicted that a boarding event is to occur. For example, asillustrated in FIG. 17A(a), 17A(b), the learning data unit 130 mayrecognize another boarding event, which indicates that passengersdifferent from passengers included in the boarding event board thevehicle, at a boarding time included in the boarding event.

In this example, the learning data unit 130 may re-recognize a boardingevent. More specifically, the learning data unit 130 may generate a newboarding event that includes currently-boarded passengers and eachboarding time. Then, the learning data unit 130 may infer a boardingpurpose for the new boarding event and extract at least one controlinformation from boarding status information on the newly-boardedpassenger to meet the inferred boarding purpose. Thereafter, thelearning data unit 130 may perform a control of the vehicle according tothe extracted at least one control information.

An example of a vehicle control method according to an addition orexclusion of passengers will be now be described. As illustrated in FIG.17A(a), 17(c), when several passengers of a plurality of expectedpassengers do not board the vehicle, the learning data unit 130 maycontrol the controller 180 to not execute the vehicle control accordingto control information related to the several passengers. In thisexample, the learning data unit 130 may execute a vehicle control basedon control information related to the boarded passengers.

Alternatively, as illustrated in FIG. 17B(a), 17B(c), the learning dataunit 130 may detect a newly-boarded passenger in addition to passengersincluded in a boarding event. In this example, the learning data unit130 may set boarding seat information from boarding status informationregarding the new passenger, and extract control information related tothe boarding seat.

As another alternative, although not illustrated, the learning data unit130 may detect a newly-boarded passenger other than passengers includedin a boarding event. In this example, the learning data unit 130 maydetermine that it is a dangerous situation, and output notificationinformation informing the dangerous situation in a preset manner. Theoutput method of the notification information may be similar to thatillustrated in FIGS. 13A(a), 13A(b), and 13B.

When a plurality of passengers board a vehicle, environmentalinformation optimized for each passenger can be provided, therebyimproving boarding convenience of the passengers. In addition, bypredicting passengers of a vehicle through various information, theenvironment of the vehicle can be preset even without a separate usercontrol command. Thus, the user can be provided with an environmentoptimized for him or her when the user gets in the vehicle.

Embodiments of the present invention can be implemented ascomputer-readable codes in a program-recorded medium. Thecomputer-readable medium may include all types of recording devices eachstoring data readable by a computer system. Examples of suchcomputer-readable media may include hard disk drive (HDD), solid statedisk (SSD), silicon disk drive (SDD), ROM, RAM, CD-ROM, magnetic tape,floppy disk, optical data storage element and the like. Also, thecomputer-readable medium may also be implemented as a format of carrierwave (e.g., transmission via an Internet). The computer may include thecontroller 180 of the terminal. Therefore, it should also be understoodthat the above-described embodiments are not limited by any of thedetails of the foregoing description, unless otherwise specified, butrather should be construed broadly within its scope as defined in theappended claims, and therefore all changes and modifications that fallwithin the metes and bounds of the claims, or equivalents of such metesand bounds are therefore intended to be embraced by the appended claims.

What is claimed is:
 1. An electronic device, comprising: a communicationunit configured to communicate with an external device; and a controllerconfigured to: obtain conversation information between the electronicdevice and the external device through the communication unit; analyzethe conversation based on a learning algorithm; identify a predictedplurality of users to be located in a specific space at a specific timebased on a result of the analyzed conversation, wherein the plurality ofusers include a first user of the electronic device and a second user ofthe external device; predict a use purpose of the specific space by theplurality of users at the specific time based on characteristicinformation of each of the plurality of users and the analyzedconversation; transmit, via the communication unit, notificationinformation related to the specific space to each of a plurality ofdevices corresponding to the plurality of users to guide the pluralityof users to the specific space at the specific time, wherein thenotification information is transmitted to each of the plurality ofdevices at a different time based on a respective location of each ofthe plurality of users and a location of the specific space; andgenerate a control command for a device associated with the specificspace at the specific time to function according to the predicted usepurpose of the specific space.
 2. The electronic device of claim 1,wherein the controller is further configured to: identify a presentplurality of people located within the specific space at the specifictime, wherein the present plurality of people does not match thepredicted plurality of users; and determine a new use purpose of thespecific space based at least on characteristic information ofadditional persons who are present that were not included in thepredicted plurality of users.
 3. The electronic device of claim 2,wherein the controller is further configured to identify the predictedplurality of users to be located within the specific space according toa message received from the external device through the communicationunit.
 4. The electronic device of claim 3, wherein the controller isfurther configured to: receive, via the communication unit, a pluralityof schedule information from the plurality of electronic devices throughthe communication unit, and generate schedule information related to thespecific space based on the received plurality of schedule information.5. The electronic device of claim 4, wherein the controller is furtherconfigured to: generate the control command based on the characteristicinformation related to each of the predicted plurality of users and thegenerated schedule information related to the specific space.
 6. Theelectronic device of claim 2, further comprising a camera configured tocapture an image of the specific space, wherein the controller isfurther configured to: identifying each of the present plurality ofpeople located within the specific space based on the captured image. 7.The electronic device of claim 1, wherein the predicted use purpose isamong a plurality of different use purposes, each having a differentcontrol command for the at least one device.
 8. The electronic device ofclaim 1, wherein the controller is further configured to extract commonelements from the characteristic information related to each of thepredicted plurality of users, wherein the use purpose is predicted basedon the extracted common elements.
 9. The electronic device of claim 1,further comprising a memory, wherein the controller is furtherconfigured to: learn the characteristic information related to each ofthe plurality of users; and store in the memory, the learnedcharacteristic information related to each of the plurality of users.10. The electronic device of claim 1, wherein the characteristicinformation related to each of the plurality of users comprises at leastone of biometric information, behavior information, history informationrelated to the specific space, or companion information of personslocated together within the specific space.
 11. The electronic device ofclaim 1, wherein the device is located at the specific space.
 12. Theelectronic device of claim 1, wherein the control command is generatedaccording to a combination of the characteristic information related toeach of the predicted plurality of users.
 13. An electronic device forassisting in operation of a vehicle, the electronic device comprising: acommunication unit configured to communicate with an external device, acontroller configured to: obtain information transmitted between theelectronic device and the external device through the communicationunit; analyze the transmitted information based on a learning algorithm;identify a predicted plurality of passengers to board the vehicle basedon a result of the analyzed transmitted information, wherein theplurality of passengers include a first user of the electronic deviceand a second user of the external device; predict a purpose of a trip ofthe vehicle based on characteristic information of each of the pluralityof passengers and the analyzed transmitted information; transmit, viathe communication unit, notification information related to boarding thevehicle to each of a plurality of devices corresponding to the predictedplurality of passengers to guide the plurality of passengers in boardingthe vehicle; and generate a control command for an aspect of the vehicleto function according to the predicted purpose.
 14. The electronicdevice of claim 13, wherein the controller is further configured to:identify a present plurality of passengers who have boarded the vehicle,wherein the present plurality of passengers does not match the predictedplurality of passengers; and determine a new purpose of the trip basedat least on characteristic information of additional persons who arepresent that were not included in the predicted plurality of users. 15.The electronic device of claim 13, wherein the controller is furtherconfigured to: set seat designations for each of the predicted pluralityof passengers prior to boarding; and generate a control command foradjusting a setting associated with a seat based on a detected state ofa particular passenger of the predicted plurality of passengersdesignated to the seat.
 16. The electronic device of claim 13, whereinthe controller is further configured to: set seat designations for eachof the predicted plurality of passengers prior to boarding; and generatea control command for the vehicle to output indicators of the seatdesignations prior to boarding.